Web Survey Bibliography
Trained respondents may answer questions differently than those with little or no experience in a panel. This can result in different responses with regard to content (e.g. because of increasing knowledge on topics) as well as the procedure (question-answering process). Having two panels—a trained one and a completely fresh one —created a unique opportunity for analysing panel conditioning effects (both panels are recruited via a probability sample of the Dutch population). Due to procedural learning, trained respondents may react differently to web survey design choices than inexperienced respondents. Because of their experience they may be able to process more information on a screen, e.g. make fewer errors when more items are placed on a single screen. In addition, they may be less sensitive to social desirability bias and less reluctant to select a response category that seems unusual in the range of responses. They also may be used to a particular question layout so that changing that layout (e.g. from disagree-agree to agree-disagree) may not be noticed. We investigated whether there are differences in design effects between trained and fresh respondents using a questionnaire consisting of three experiments. In the experiments we varied the number of items on a screen, the choice of response categories, and the layout of a five point rating scale. We found that trained respondents were somewhat more sensitive to satisficing. Besides procedural knowledge, trained respondents can develop knowledge on topics addressed in previous surveys. To find out the relation between question type and panel conditioning, trained and fresh respondents answered questions with different question types. We used nine questions on two different subjects: food infection and old-age pensions. These subjects were presented to the trained panel multiple times, so we thought these would be the most sensitive to bias due to panel conditioning. The results show that panel conditioning only arises in knowledge questions. Questions on attitudes, actual behaviour, or facts were not sensitive to panel conditioning. Panel conditioning in knowledge questions was restricted to less-known subjects (more difficult questions), suggesting a relation between panel conditioning and cognition.
Conference homepage (abstract)
Web survey bibliography - Toepoel, V. (27)
- Using experts’ consensus (the Delphi method) to evaluate weighting techniques in web surveys not...; 2017; Toepoel, V.; Emerson, H.
- Data chunking for mobile web: effects on data quality; 2017; Lugtig, P. J.; Toepoel, V.
- Mobile-only web survey respondents; 2016; Lugtig, P. J.; Toepoel, V.; Amin, A.
- Online Surveys are Mixed-Device Surveys. Issues Associated with the Use of Different (Mobile) Devices...; 2016; Toepoel, V.; Lugtig, P. J.
- Doing Surveys Online ; 2016; Toepoel, V.
- The Effects of Adding a Mobile-Compatible Design to the American Life Panel; 2015; Toepoel, V.; Lugtig, P. J.; Amin, A.
- Higher Item Nonresponse Rates Caused by Slider Scales in Web Surveys; 2015; Toepoel, V.; Funke, F.
- Coding Surveys on their Item Characteristics: Reliability Diagnostics; 2015; Bais, F.; Schouten, B.; Toepoel, V.
- Investigating Response Quality in Mobile and Desktop Surveys: A Comparison of Radio Buttons, Visual...; 2014; Toepoel, V.; Funke, F.
- Informing panel members about study results; 2014; Scherpenzeel, A., Toepoel, V.
- Mixed-devices in a probability based panel survey. Effects on survey measurement error; 2014; Toepoel, V., Lugtig, P. J.
- Panel Conditioning in Difficult Attitudinal Questions; 2013; Binswanger, J., Schunk, D., Toepoel, V.
- Mobile devices a way to recruit hard-to-reach groups? Results from a pilot study comparing desk top...; 2013; Toepoel, V., Lugtig, P. J.
- Effects of Incentives in Surveys; 2012; Toepoel, V.
- Building Your Own Online Panel Via E-Mail and Other Digital Media; 2012; Toepoel, V.
- Recruiting A Probability Sample For An Online Panel: Effects Of Contact Mode, Incentives, And Information...; 2012; Scherpenzeel, A., Toepoel, V.
- How Visual Design Affects the Interpretability of Survey Questions; 2011; Toepoel, V., Dillman, D. A.
- Panel Recruitment via Facebook; 2011; Toepoel, V.
- Nonparametric Tests of Panel Conditioning and Attrition Bias in Panel Surveys; 2011; Das, M., Toepoel, V., van Soest, A.
- Response Quantity, Response Quality, and Costs of Building an Online Panel via Social Contacts.; 2011; Toepoel, V.
- Can Verbal Instructions Counteract Visual Context Effects in Web Surveys?; 2011; Toepoel, V., Couper, M. P.
- Words, Numbers and Visual Heuristics in Web Surveys: Is There a Hierarchy of Importance?; 2009; Toepoel, V., Dillman, D. A.
- Pictures in Web Surveys; 2009; Toepoel, V., Couper, M. P.
- Panel Conditioning in Web Surveys: A Comparison between Trained and Fresh Respondents; 2009; Toepoel, V., Das, M., van Soest, A.
- Visual Heuristics and Answer Formats in Rating Scales; 2009; Toepoel, V. Dillman, D. A.
- Design effects in web surveys: comparing trained and fresh respondents; 2008; Toepoel, V., Das, M., van Soest, A.
- Can I use a panel? Panel conditioning and attrition bias in panel surveys; 2007; Das, M., Toepoel, V., van Soest, A.